Transparent CPU-GPU Collaboration for Data-Parallel Kernels on Heterogeneous Systems

This paper presented the single kernel multiple devices (SKMD) system, a framework that transparently orchestrates collaborative execution of a single data-parallel kernel across multiple asymmetric CPUs and GPUs. SKMD is an abstraction layer located between applications and the OpenCL library. It uses OpenCL as the intermediate language. SKMD transparently partitions an OpenCL kernel across multiple devices being aware of the transfer cost and performance variation on the workload, launches parallel kernels, and merges the partial results into the final output automatically. Their system not only eliminates the tedious process of reengineering applications when the hardware changes, but also makes efficient partitioning decisions based on application characteristics, input sizes, and the underlying hardware. The dynamic compiler of SKMD has three main components:Kernel Transformer, Buffer Manager and Partitioner. The kernel transformer changes the original kernel to Partition-Ready kernel, which enables the kernel to work only on a subset of work-groups. After kernel transformation, the buffer manager performs static analysis on kernels to determine memory access pattern of each work-group. If memory access range of each work-group can be analyzed statically, the buffer manager will transfer only necessary data back; if memory access range cannot be analyzed, entire input should be transferred to each device and output must be merged. In order to merge irregular locations of output from different devices, the kernel transformer generates Merge Kernel, and SKMD launches it on CPU device.